Skip to content

Feat blindsage tft images - #440

Open
cablesandcoffee wants to merge 76 commits into
mainfrom
feat-blindsage-tft-images
Open

cablesandcoffee wants to merge 76 commits into
mainfrom
feat-blindsage-tft-images

Conversation

@cablesandcoffee

Copy link
Copy Markdown
Collaborator

No description provided.

tpanajott and others added 30 commits August 20, 2026 20:45
* Add media player entity to NSPanel protobufs

Add a MediaPlayer entity state (playback state, title, artist, volume,
mute, optional source volume and capability flags), a MediaPlayerCommand
for panel to manager control, and ENTITY_TYPE_MEDIA_PLAYER for entity
page slots.

Also add the REBOOTING panel state that the firmware protobuf already
carries, so regenerated firmware files keep it.

Generated with protoc 27.0 and protoc-gen-c 1.4.1.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7aYyRjzk53Sd7gxF46E3u

* Add album art URL to the media player entity state

The manager can fetch the album art from Home Assistant, convert it into
a format the panel can draw, and serve it. The new field tells the panel
where to fetch it. The URL changes whenever the art changes, so the panel
can compare it with the last URL it drew to tell when to fetch again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* Add Home Assistant media player entities

Media players can be placed on a room's entities page. The manager follows
the Home Assistant media_player entity and publishes its state to the
NSPanel as an NSPanelEntityState protobuf. Media player commands from the
panel are forwarded to Home Assistant.

Some setups, such as multi-room audio matrices, have a separate volume for
the source feeding a player. How that is exposed in Home Assistant differs
between setups, so a global setting selects a source volume strategy:

- none (default): no source volume.
- player_attributes: the player reports the source volume and the source
  entity ID as state attributes. Volume changes go to the source entity.
- source_entity: the player reports the source entity ID as a state
  attribute and the source's own volume_level is used.

Media players are added through a new REST endpoint for now, the web UI
will follow separately.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7aYyRjzk53Sd7gxF46E3u

* Serve media player album art to NSPanels

Media players now save the "entity_picture" attribute from Home Assistant
and give the NSPanel an album_art_url pointing at the Nextion image
server on the manager. The URL includes a hash of the source, so it
changes whenever the album art does.

When a panel requests the image, the manager downloads it from Home
Assistant (once, cached until the art changes) and converts it for the
panel: center cropped and scaled to the requested size (default
100x100), returned as raw little-endian RGB565 pixels. format=png
returns the same image as a PNG for checking the conversion in a
browser.

The Nextion image server was removed from the build because ImageMagick
broke static linking of the release binary. It is back, using the
header-only stb library for decoding and resizing instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Validate media player entity requests

- Only accept PUT. Other methods used to create or update entities too.
- Only update existing media players. Passing the ID of another type of
  entity used to overwrite it.
- Check that the entities page belongs to the room and is not a scenes
  page, that the slot is within the page and that it is free.
- Return 400/404 for bad or unknown IDs instead of 500.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Configure source volume per media player instead of globally

The source volume strategy and the attribute names it reads were global
settings, but they describe how one Home Assistant integration exposes
the source feeding a player. A setup with, say, an audio matrix and an
AVR could not describe both, and nothing in the settings UI could have
made that any better.

They are now part of the media player's entity_data, set through the
same PUT /rest/entities/media_players request that creates the player.
All three fields are optional; a player without them keeps its own
volume only.

The endpoint rejects a strategy that is missing the attributes it needs,
which the manager could previously only log and fall back to no source
volume for, leaving the user with no indication of why.

On the manager side HomeAssistantSourceVolumeStrategy::create() reads
the entity's data rather than MqttManagerConfig, so the three
MQTT_MANAGER_SETTING entries are gone. Loading the config now returns
the entity_data for the strategy to be built from, which also fixes the
ordering: the strategy used to be created before the config it depends
on was read.

The web UI for all of this is still missing, see MEDIA_PLAYER_TODO.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Address review comments on media players and album art

- Only detach from and reattach to Home Assistant when the media
  player's entity ID changes, and only rebuild the source volume
  strategy when its settings change. A rebuilt strategy starts from the
  last player attributes, since no fresh state is requested otherwise.
- Explicitly reject OpenHAB media players, which are not implemented,
  instead of treating them as unknown controllers.
- Warn about unknown playback actions. NONE is expected, it is sent with
  volume and mute changes.
- Catch exceptions in the Nextion image server request handler and
  respond with a 500 instead.
- Cache converted album art per media player, size and format, so panels
  requesting the same image do not each decode and scale it again.
  get_album_art() returns a shared pointer, which also tells the cache
  when the art has changed.
- Allow images up to 1024 px per side, for format=png on displays larger
  than the NSPanel.
- Use make_shared to create Home Assistant media players, and uint16_t
  for the manager port.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add a per media player volume step

Some players need bigger or smaller volume steps than others, an Android
TV box might have 10 volume levels where a receiver has 80, so the step
is set per media player rather than globally.

- volume_step is an optional integer between 1 and 100 (%) in
  PUT /rest/entities/media_players. A new media player defaults to 5,
  and updating an existing one without it keeps its current step.
- The MQTTManager reads it from entity_data, falling back to 5 when it
  is missing or invalid, and sends it to the panel.
- NSPanelEntityState.MediaPlayer gains volume_step (field 17). The same
  step applies to the volume and the source volume. Regenerated with
  protoc 27.0.

The firmware's volume up/down buttons and the web UI field are still to
do, see MEDIA_PLAYER_TODO.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Remove MEDIA_PLAYER_TODO.md

It is a local working note rather than part of the change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…tates would get reset when reloading manager config
* First work on first page react setup

* Initial work on react first page

* Almost all functionality is now replicated and improved on from the HTMX based solution

* Pre merge with devel

* Fixed bugs

* There is now a global toast/notify framework for react to use

* React now uses RXStomp as a backend for handling STOMP

* Finishing touces and also removing old HTMX first page setup

* First page now uses react MDI icons instead of web font.

* Fixed typos

* Initial setup is now performed using React instead of bodged HTMX guide

* NSPanel first page view now handles rebooting state correctly

* Updated icons to use mdi SVGs via React and laos updated some error messages

* Updated nspanel firmware protobuf files

* Set fixed height of NSPanel boxes on first page

* Pre devel merge

* Fixed an error where resolved warnings would not be removed from web ui

* Editing entities will now show error in notification instead of in developer console

* Cleanup of old HTMX files and switched nspanel status and actions in the edit nspanel page to be a react component

* NSPanel warnings on edit_nspanel page is now a react component. Also fixed animation bug with progressbar on edit_nspanel page during updating

* Updated style of progressbar during updating in edit_nspanel sidebar

* Fixed a bug where an empty nspanel warnings container would be shown if no warnings were active instead of hiding the container

* NSPanel logs view is now a react component and it's also possible to download nspanel logs now.

* Fixed issues cuasing bun build to fail

* Fixed a bug that caused chrome browsers to fail to connect to stomp socket via react components

* Fixed a bug where debouncing of state update events in rooms were not working correctly

* Updated CSS

* Updated theme

* Static grid complete

* Background animation of boxes on NSPanelBox now follows cursor with a gradiant

* Small visual improvments

* Will now set permissions on manifest.json as to be able to merge changes from different branches

* Also set permissions on dir

* Fixed tiles animation visibility

* Started work on make eidt_room a full react page

* Moved stop into docker dir

* It's now possible to change room name and temperature sensor

* Manager configured NSPanel friendly name is now repected and retained across NSPanel reboots

* Updated gitignore

* Fixed openhab lights not loading the correct key for openhab item for switches in the new React setup

* Fixed multiple react bugs. Also fixed a bug that caused all panels to become of state "offline" when the manager reloaded.
#387)

* Add port, container name and seed path options to the dev build script

docker-build_and_run_dev.sh always ran on port 8000 as a container named
nspanelmanager, so a second dev stack could not run next to the first.

- --external-port: host port mapped to the container's port 8000.
- --container-name: name of the running container.
- --config-seed-path: where to copy db.sqlite3 and secret.key from when
  data/ does not have them yet. Defaults to the current directory, as
  before.

--target-platform only shifted once, so its value was parsed as the next
option. It now shifts past the value like the new options.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Fail fast when --config-seed-path is missing the DB or secret key

An explicit seed path with a typo previously fell through silently and
started the stack with an empty database. Now the script exits before
building if either file is missing, and warns when data/ already has
files that the seed will not overwrite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
… install

#391 replaced the htmx first-run wizard with the React InitialSetup and
removed its htmx_initial_setup_* URLs, but base.html still included
modals/initial_setup/welcome.html whenever manager_address is empty.
That template reverses the removed URLs, so on a fresh install every
page, including the index that shows the new wizard, raised
NoReverseMatch and returned 500.

Remove the include and the now-unreferenced modals/initial_setup
templates, and the index.html script tag for initial_setup.js, which
#391 deleted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Add MQTTManager unit test harness

Wire GoogleTest into the CMake build behind an NSPM_BUILD_TESTS option and add
a tests/ directory with its own test executable. The existing `if (TEST_MODE==1)`
block never ran, so the inline tests had never been built or discovered.

- tests/: main.cpp initialises the (TEST_MODE) database; test_helpers.hpp has a
  ScopedEntity RAII helper and builds entity_data exactly as Django's rest.py
  writes it; light_config_test.cpp checks lights load placement, capabilities
  and controller from Django-shaped rows.
- The inline TEST_MODE tests in the Config and Light libraries are linked in
  whole so they register with GoogleTest (21 tests in total).
- run_tests.sh builds and runs everything in Docker, caching Conan packages and
  build output in named volumes.
- compile_mqttmanager.sh --test now sets the CMake option instead of using sed.

Fixes found by turning the tests on:
- Remove the copies of the Home Assistant light tests at the end of both
  thermostat sources; they did not compile and duplicated test names.
- Add the missing OPENHAB_RGB_CHANNEL_NAME entry to the settings key map
  (caught by MqttManagerConfigTest.verify_all_settings_exists_and_have_db_key).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Run the MQTTManager tests in GitHub Actions

Runs docker/MQTTManager/run_tests.sh on pull requests and pushes that touch
docker/MQTTManager. The Conan package cache is a host directory saved with
actions/cache (keyed on conanfile.py and the test Dockerfile), so only the
first run pays for building the dependencies from source.

run_tests.sh takes NSPM_TEST_CONAN_CACHE / NSPM_TEST_BUILD_DIR to override
its Docker volumes, and the container now prunes Conan's build and source
trees after installing to keep the cache small.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Report MQTTManager test results in the GitHub Actions job summary

run_tests.sh writes gtest JUnit XML to NSPM_TEST_RESULTS_DIR when it is set,
and CI publishes it with action-junit-report. annotate_only is used because
creating a check run needs checks: write, which pull requests from forks
don't get.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add MQTTManager tests for switches, buttons, scenes, thermostats, media players and the EntityManager

In test builds, HomeAssistantManager, OpenhabManager and MQTT_Manager record what they send,
so tests can check the exact service calls and MQTT messages without a live connection.
HomeAssistantManager::test_process_event() delivers an event through the real observer routing.
All of it is behind TEST_MODE, so production builds are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Initialise the room temperature provider

Room::_room_temp_provider was never initialised, so loading a room without a temperature
sensor logged "Got unknown temperature provider ... Provider '<random number>'", and if the
garbage happened to match a controller it detached an observer that was never attached.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Check for the NSPM controller in the NSPMButton constructor

The check was copied from HomeAssistantSwitch, so every NSPM button logged "HomeAssistantSwitch
has not been recognized as controlled by HOME_ASSISTANT" when it was loaded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add OpenHAB switch, scene and thermostat tests

Adds a TEST_MODE hook that feeds a raw websocket message through
OpenhabManager's parser, and one that points the REST API at an address
without opening a websocket. Tests use a small fake HTTP server on
127.0.0.1 for the REST calls (initial item state, rule runs).

Known bugs are pinned as DISABLED_ tests:
- Thermostat reads openhab_preset_item/openhab_swing_item/
  openhab_swingh_item but Django writes *_mode_item, so presets and
  swing are sent to openhab/items//command and never followed.
- The current temperature item is never attached, and its callback
  checks the target temperature item.
- The initial target temperature fetched over REST is rounded.
- OpenhabScene::activate passes a dangling c_str() as the
  Authorization header.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add NSPanel config and command tests

Cover the NSPanelConfig the manager publishes (defaults, panel settings,
temperature calibration, screensaver fallbacks, button modes, thermostat
limits, relay group bindings, room infos and locking, reload, unknown room,
denied panel) and the command paths (reboot topics, MQTT payload buttons,
detached buttons toggling entities and activating scenes).

Adds TEST_MODE hooks to count MQTT and CommandManager callbacks.

Pinned as DISABLED_ (KNOWN BUG):
- a panel that is neither accepted nor denied reports WAITING, not
  AWAITING_ACCEPT
- ~NSPanel leaves its CommandManager callback and log/legacy status
  subscriptions attached

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Send the room temperature sensor topic to panels again

d8220f6 moved the thermostat temperature limits in NSPanel::send_config()
and dropped the line that sets inside_temperature_sensor_mqtt_topic along
with them. Since then panels always show their built-in sensor on the
screensaver, even when their room has a temperature sensor configured.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Don't crash on a thermostat temperature limit that isn't a number

The thermostat limits are free text in the web interface, and
send_config() parsed them with std::stoi. d8220f6 stopped empty values
being saved, but any other non-numeric value still threw
std::invalid_argument, which reload_config() does not catch (it only
catches std::system_error). Parse the limits safely, logging an error and
sending 0 for a bad value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Initialise NSPanel humidity, pressure and relay members

- _humidity and _pressure were never initialised, so the web interface
  was sent whatever was in memory until the panel's first status report.
  Reset them with the other readings when the panel is offline.
- _relay1_state, _relay2_state and _register_relay1/2_as_light were never
  initialised. Default them to false.
- The relay 2 "Will register" debug log printed relay 1's setting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Keep panels that haven't been accepted in AWAITING_ACCEPT

Since #391 the NSPanel constructor sets WAITING whenever the panel has a
config topic, which pending panels have too, so it overwrote the
AWAITING_ACCEPT state set by reload_config(). A newly discovered panel was
reported to the web interface as "waiting" with "accepted": true. Only set
WAITING for panels that are not awaiting accept, and drop the duplicated
block that set OFFLINE just before it.

Accepting a panel only worked because of that bug: register requests from
panels in AWAITING_ACCEPT or DENIED are ignored, and nothing moved an
accepted panel out of those states. reload_config() now moves a panel
that has just been accepted to WAITING.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Detach all NSPanel callbacks when a panel is destroyed

~NSPanel() never detached its CommandManager callback, and
reset_mqtt_topics() missed the nspanel/<mac>/log subscription and the
legacy nspanel/<name>/status and status_report subscriptions, which were
built inline and never stored. After a panel was deleted, the next button
press from any panel, or the next message on one of those topics, called
into the destroyed NSPanel.

Keep those topics in members so they can be detached, which also moves
the legacy subscriptions over when a panel is renamed instead of leaving
the old name subscribed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Round decimal thermostat limits instead of truncating them

std::stoi stopped at the first character it couldn't read, so "21.5" was
silently sent as 21 and "21abc" as 21, and the "not a whole number" error
was only ever logged for values that weren't numbers at all. Parse the
whole value instead:
- not a number (including trailing text): error, send 0
- out of range: error, send 0
- a decimal: round to the nearest degree and log a warning

ScopedErrorLog can now record warnings too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Initialise the OpenhabThermostat debounce timestamps

OpenhabThermostat ignores an OpenHAB event for an item unless at least a
second has passed since the item last changed, but the _last_*_change
timestamps it compares against were never initialised. When the memory
held a large value, target temperature, mode, fan, preset or swing
updates from OpenHAB were silently ignored for that thermostat. CI hit
this in OpenhabThermostatTest.follows_target_temperature_and_mode_from_openhab.
Set them to 0 in the constructor, as OpenhabSwitch and OpenhabLight do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Initialise the OpenhabLight group event state

The _openhab_group_*_item_state_changed_event_thread_running flags were
never initialised. If one started out true, the thread that processes
that group's events (brightness, colour temperature or RGB) was never
started, so the light ignored those OpenHAB group events. Set them to
false in the constructor, along with the matching group event timestamps
(always written before they are read today, but uninitialised).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Test Home Assistant script activation from panels end to end

The scene tests already check the nspanelmanager context a script gets
when activate() is called. These cover the paths that call it:
- tapping a scene or script on a panel's page (ToggleEntityFromEntitiesPage)
  sends the panel's room and the script's room, or only the panel's room
  for a global script, and still runs from an unknown panel without a
  triggering room
- the triggering room follows the panel when it moves room, and a renamed
  room's new name is sent
- a detached button runs a script with the full context, or turns on a
  scene

Adds a TEST_MODE hook to hand EntityManager a panel command without
starting EntityManager::init()'s threads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Don't run static destructors when the test binary exits

Every room starts a detached thread that waits on the room's mutex and
condition variable forever. Since the tests started creating rooms, the
binary segfaults (exit code 139) after all tests have passed: exit
destroys EntityManager's static room list while those threads are still
waiting. The manager never exits this way, so leave the threads alone
and exit with std::_Exit once the results are written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Report crashes in the test binary with the test name and a backtrace

CI has seen the test binary segfault part-way through a run, which only
showed up as exit code 139 with the last buffered output missing. Make
stdout line-buffered, and on a fatal signal print the running test and a
demangled backtrace before re-raising. Link with -rdynamic so frames in
the executable are named.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Lock the MQTT retained message buffer

MQTT_Manager::publish() and clear_retain() wrote to _mqtt_retain_buffer,
a std::unordered_map, without holding any lock. Room status threads,
entity updates and panel reloads publish retained messages from different
threads, so concurrent inserts and erases could corrupt the map during a
rehash. The test binary crashed this way in CI: a room's status thread
published its retained state while a panel was loading and clearing its
Home Assistant discovery topics.

Give the buffer its own mutex, and replay it from a copy on reconnect,
since publish() writes to it. The reconnect also sent and erased the
buffered messages without _mqtt_client_mutex, which publish() and
clear_retain() hold when adding to that list; take it there too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Hold _rooms_mutex when changing or walking EntityManager's room list

load_rooms() erased removed rooms from _rooms and sorted it without
_rooms_mutex, and update_all_rooms_status() walked it without the lock,
while get_room(), get_all_rooms() and the room command handlers lock it.
A config reload could reallocate or reorder the list under a reader.
Take the lock for the erase and the sort, and have the 'All rooms'
thread work on a copy taken under the lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Remove the unused openhab_rgb_channel_name setting

It was left over from an early plan for OpenHAB RGB lights and nothing reads it.
Drop it from the MQTTManager settings enum and key map, the Django settings view
and the React settings store, rather than giving it a key map entry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Build the tests' fake HTTP server on ix::HttpServer

Replace the hand-written socket and HTTP parsing code with ixwebsocket's
HttpServer, which MQTTManager already depends on and uses for the Nextion image
server. ix::HttpServer can't bind to port 0, so the fake tries ports from 18800
until one is free.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The dev container bind-mounts the checkout's MQTTManager/ at
/MQTTManager/, and the container's Conan config stores packages under
/MQTTManager/conan_cache/. Every checkout or git worktree therefore
grows its own multi-gigabyte cache inside the source tree.

If NSPM_CONAN_CACHE is set, mount that host directory over
/MQTTManager/conan_cache/ so all checkouts share one cache. The path
inside the container is unchanged, which matters because Conan 2
records absolute paths in its cache database. With the variable unset,
behaviour is as before.

sudo drops the variable from the environment by default, so it must be
kept explicitly when the script is run under sudo.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…iling (#399)

The web interface triggers a reload on every save, so a reload's reads often
run while that write is still committing. MQTTManager opened SQLite without a
busy timeout, so those reads failed at once with "database is locked", the
settings getters returned empty strings, and NSPanel's config builder aborted
the process on std::stoi(""). Deleting a light and then adding another in quick
succession was enough to trigger it.

Give every connection a 20 s busy timeout, matching the web interface's.
`database` is a static in a header, so each translation unit has its own
storage; the timeout is set where each copy is built rather than in init(),
which only ever configured main.cpp's copy.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Send per-event buzzer sounds to panels in NSPanelConfig

Add a repeated Sound field to NSPanelConfig that maps panel events
(TOUCH, SCENE_ACTIVATED, SCENE_SAVED) to RTTTL melodies, for the panel
buzzer support added to the firmware in NSPanelManager-firmware#29.

The sounds come from three new global settings, buzzer_sound_touch,
buzzer_sound_scene_activated and buzzer_sound_scene_saved, which a
panel setting with the same name overrides. They default to empty,
which sends no entry and leaves the event silent, so panels behave as
before until a sound is configured. There is no UI for them yet; set
them through POST /rest/settings.

Generated with protoc 27.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Test the buzzer sounds sent in NSPanelConfig

Covers a panel with no buzzer settings, the global sound for each event,
and panel settings overriding the global ones, including an empty panel
setting silencing an event that has a global sound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Add a Django test harness and initial REST API tests

Adds web/tests/ with a shared NSPMTestCase harness and 97 tests covering
rooms, entity create/edit (lights, switches, buttons, thermostats, scenes)
and entity/page layout, exercised through the same REST endpoints and
payloads the React UI uses.

The harness:
- runs each test against an isolated in-memory database
- replaces the MQTTManager process-control hooks with a recorder, so tests
  can assert the manager was told to reload and can never signal a real
  nspm_mqttmanager process on the host
- fakes Home Assistant (at the HTTP layer) and OpenHAB, and fails any other
  network request
- provides assertLoadableByManager(), which checks saved rows have the
  shape MQTTManager parses when it reads the database

Known bugs and validation gaps are pinned as @expectedfailure tests with a
KNOWN BUG / KNOWN GAP comment, so the suite passes today and flags when a
fix lands (remove the decorator then).

settings.py now reads the data directory from NSPM_DATA_DIR (default
/data, unchanged in the container) so the app can start outside Docker.

Run with docker/web/run_tests.sh; CI runs it on PRs touching docker/web.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Expand Django test coverage

Adds 113 tests (210 total) covering the rest of the web app:

- test_pages: every full page and htmx partial renders, including on a
  fresh install, and the temperature-sensor pickers route to the
  configured source
- test_settings: settings form (secret handling, trailing slashes,
  defaults), weather & time, theme, the first-run wizard, the settings REST
  endpoints and web.settings_helper
- test_panels: accept/deny/delete/unblock, the per-panel settings form and
  the legacy api/ panel endpoints
- test_relay_groups: form views, htmx endpoints and the REST listing
- test_files: firmware/LittleFS/TFT upload, checksum and download,
  including which TFT each panel type gets and a contract test for the
  exclusive Range end the panel firmware depends on
- test_misc: messages, HA/OpenHAB entity lookups, hostname lookup and
  read-only listings

New known issues are pinned with @expectedfailure as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Make the MQTTManager settings endpoint's banned-key check case-insensitive

banned_setting_keys lists secrets in upper case ("HOME_ASSISTANT_TOKEN"),
but settings are stored in lower case ("home_assistant_token") and the
comparison was case-sensitive, so /rest/mqttmanager/settings returned the
MQTT password and Home Assistant/OpenHAB tokens it was meant to withhold.

Nothing in the repo calls this endpoint (MQTTManager reads settings from
the database directly), so refusing these keys breaks no caller.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Tell MQTTManager to reload after deletes and edits that skipped it

These views changed config MQTTManager reads from the database but never
called send_mqttmanager_reload_command(), so panels kept the old config
until something else triggered a reload:

- DELETE /rest/scenes/<id>
- PUT /rest/entities_pages/<id> (page resize)
- htmx/nspanels/<id>/delete
- htmx relay group save and delete

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Make /rest/rooms create and delete match the UI's room views

- POST /rest/rooms now creates the room's entities and scenes pages, as
  save_new_room does, instead of leaving them to the next restart. The
  per-room part of create_entity_pages_for_all_rooms() is split out as
  create_entity_pages_for_room() so both paths share it.
- The name must be 1-30 characters (Room.friendly_name's max_length,
  which SQLite does not enforce).
- DELETE /rest/rooms/<id> moves the room's panels to another room, as
  delete_room does, and tells MQTTManager to reload. Creating a room now
  reloads too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Apply PUT /rest/rooms/<id>/entities/order atomically

Each entry was saved as it was read, so an unknown id part-way through
the list returned an error but left the entries before it already moved,
a half-applied layout. Run the updates in one transaction so a failed
request changes nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Validate where entities and scenes are placed on a page

The /rest/entities/* endpoints stored room_view_position and
entities_page_id as given, so the UI (or any other caller) could:

- put two entities in the same slot;
- put one in a slot past the end of its page, where the panel never
  shows it;
- put an entity on another room's page.

get_placement_error() checks all three and the endpoints return 400
with a message. Only a new or changed placement is checked, so an
entity already saved in a bad slot can still be edited in place.

PUT /rest/rooms/<id>/entities/order now also rejects slots past the end
of a page. It does not check for occupied slots, because a swap moves
two entities through each other's slot in one request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Reject entities and scenes MQTTManager cannot load

The entity endpoints stored controller and scene_type as given. MQTTManager
logs an error for an entity whose controller its class does not handle
and skips a scene with an unknown scene_type, so these saved fine but
never worked on a panel. Now rejected with 400:

- a controller the entity type does not support (e.g. an OpenHAB button);
- an unknown scene_type;
- a Home Assistant or OpenHAB scene with an empty backend_name, which
  could never be activated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Only allow 4, 8 and 12 entity pages, and do not strand entities on shrink

- Creating a page (room or global) with any other size is rejected with
  400; the panel GUI only has those three layouts.
- PUT /rest/entities_pages/<id> checks the new size the same way, and
  refuses to shrink a page while an entity or scene sits in a slot past
  the new end. Those entities stayed attached to the page but were never
  shown on the panel. The UI already only offers 4/8/12.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Reject non-PUT requests to the thermostat and scene entity endpoints

The method check in entities_thermostats() and entities_scenes() was
commented out, so a GET, POST or DELETE was handled as a save. Restore
it, and answer unsupported methods on all five /rest/entities/* save
endpoints with 405 rather than 403.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Fix three small REST bugs: settings 500, scene filter, missing checksums

- GET /rest/settings returned 500 until the Home Assistant token, the
  OpenHAB token and the MQTT password had all been saved at least once,
  because it del'd each key without checking it existed (e.g. OpenHAB
  never configured). Missing secrets now report *_set: false.
- GET /rest/scenes?scene_id=N checked for a light_id parameter, then
  filtered on scene_id, so the filter was ignored and every scene came
  back.
- The checksum_firmware, checksum_data_file and checksum_tft_file
  endpoints returned 200 with the body "None" when no image had been
  uploaded. They now return 404 with an empty body. The panel firmware
  does not check the status code, so for a panel this only changes the
  string it compares against its stored checksum.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Fix api/get_nspanel_config and remove two dead htmx partials

- api/get_nspanel_config always returned 500: it read
  button1/2_detached_mode_light, which were renamed to
  button1/2_detached_mode_entity. Use the new names, and fix the else
  branch writing -1 to "button1_detached_mode_light" instead of the
  "button1_detached_light" key the other branch sets. Kept rather than
  removed because panels on older firmware may still call it.
- Remove htmx/partial/nspanel_index_view/<id>, which always raised
  ImproperlyConfigured (nspanel_status_header has no template for that
  URL name), and htmx/partial/edit_entities_page/<id>, whose template was
  removed in the move to React, along with its save/<page_type> route and
  the edit button in the old htmx entities-page component that linked to
  it. Nothing in the current UI reaches any of them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Update tests for the React first page (#391)

- The htmx first-run wizard and the index panels partial are gone; the
  React InitialSetup now saves everything with one POST /rest/settings.
  Test that endpoint (saves, only touches submitted keys, rejects bad
  payloads, reloads once) and that the index shows InitialSetup until a
  manager address is configured.
- /rest/mqttmanager/settings (POST) was removed; keep the banned-key
  tests for the per-key GET endpoint.
- GET /rest/rooms now returns friendly_name, display_order and the
  temperature sensor fields; PUT /rest/rooms updates a room. Pin the
  display_order/displayOrder mismatch in room_put as a known bug.
- Thermostats now require use_current_temperature.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Remove the leftover htmx setup wizard that 500s every page on a fresh install

#391 replaced the htmx first-run wizard with the React InitialSetup and
removed its htmx_initial_setup_* URLs, but base.html still included
modals/initial_setup/welcome.html whenever manager_address is empty.
That template reverses the removed URLs, so on a fresh install every
page, including the index that shows the new wizard, raised
NoReverseMatch and returned 500.

Remove the include and the now-unreferenced modals/initial_setup
templates, and the index.html script tag for initial_setup.js, which
#391 deleted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Make the wizard test settings a module constant (ruff RUF012)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Report Django test results in the GitHub Actions job summary

Write JUnit XML from the test run in CI (via unittest-xml-reporting, enabled
by NSPM_TEST_JUNIT_DIR) and publish it with action-junit-report. annotate_only
is used because creating a check run needs checks: write, which pull requests
from forks don't get.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Test adding Home Assistant scripts as scenes

The React scene picker fetches /rest/home_assistant/entities and offers
the scene.* and script.* items. Check that scripts are listed there, and
that a script can be saved as a room or global scene through
/rest/entities/scenes in a form the manager can load.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Address review feedback on the Django test harness

- Remove endpoints the UI no longer uses: the htmx panel accept/deny/delete
  views (React uses /rest/nspanel/<id>, /accept and /deny) and
  /rest/mqttmanager/settings/<key> (MQTTManager reads settings from the
  database). Removing the latter also supersedes the banned-keys fix.
- Point the panel lifecycle tests at the /rest/nspanel endpoints, set up the
  denied-panels page test directly, and explain the blank-override test.
- assertLoadableByManager now checks the stored entity_data text is a JSON
  object, since that is what MQTTManager parses.
- Add snapshot()/assertOnlyChanged() and use them so edit tests fail if a
  save changes any field other than the one being edited.
- CI: also run on pushes to beta, and pin the workflow token to read-only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…onnected (#403)

* MQTTManager: don't let new messages overtake ones buffered while disconnected

Messages published while disconnected from MQTT are buffered and sent
after reconnecting. But publish() went back to sending live as soon as
the client reported connected, while the reconnect still had to replay
the retained values and subscribe to every topic (5 ms each) before it
sent the buffer. Anything published in that window went out first and
was then overwritten by an older buffered message.

On a real install, a media player's state published before the first
connection (only its ID and name, as Home Assistant had not answered
yet) was buffered, the real state was published live half a second
later, and the buffered placeholder then replaced it on the broker. The
manager saw no change to send after that, so panels showed an empty
media player for hours.

The retained replay had the same problem: it published from a copy
taken earlier, through publish(), so it could send an older value after
a newer one and write the older value back into the retain buffer.

is_connected() now also requires that the buffered messages have been
sent. Until then publish() and clear_retain() buffer, behind what is
already waiting. After connecting, the buffer is sent in order, then the
latest value of every retained topic, all under the client lock, and
only then are topics subscribed to again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Tests: stop changing the logger's sinks while Room threads log

ScopedErrorLog pushed its sink onto, and erased it from, the default
logger's sink vector. spdlog does not lock that vector, and every Room
runs a thread that logs for the rest of the run, so a capture starting
or ending while one of those threads logged could reallocate the vector
under it. That crashed a CI run in spdlog::logger::sink_it_, called from
Room::_update_room_state(), between tests.

main() now adds a single dist_sink to the default logger before any test
runs, and ScopedErrorLog adds and removes its sink through that, which
locks its own list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…406)

* MQTTManager: stop a room's status thread when the room is destroyed

Each Room started a detached thread that waited on the room's own mutex
and condition variable, and sent the room's status, forever. Nothing
stopped it, so once a room was deleted and EntityManager dropped it, the
thread kept using the freed Room: it either stayed blocked on a freed
condition variable or, if a change was waiting to settle, went on to
build and publish a status from freed memory. That could crash the
manager at an unrelated point, or publish a stale state to the deleted
room's topic just after ~Room() had cleared it.

The thread is now a std::jthread whose waits take its stop token, and
~Room() stops and joins it before anything else is torn down. Stopping
wakes the thread from either wait, so the join only waits for an update
already being sent.

The backoff used wait_for() returning no_timeout to tell that another
change had arrived. The stop token overloads take a predicate instead,
so changes are now counted and the backoff waits for the count to
change. This also stops a spurious wakeup from restarting the backoff.

EntityManager::load_rooms() now destroys removed rooms after releasing
_rooms_mutex, so that the join never waits while holding it. If a room
is ever destroyed on its own thread, which nothing does today, it logs
and detaches instead of throwing from join().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MQTTManager: drop load_rooms() change, test destroying an idle room

Revert load_rooms() to destroy removed rooms with erase/remove_if. Nothing
on a room's status thread takes _rooms_mutex, so joining it under that lock
cannot deadlock, and the extra code was not needed.

Add a test that destroys a room whose status thread is waiting for the next
change, which only the stop request can wake.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Send NSPanels the POSIX TZ rule for the manager's timezone

The manager reads the rule from the end of the configured zone's tzdata
file and sends it in NSPanelConfig.tz_posix, so a panel can render local
time from its own clock with setenv("TZ") and tzset(). Empty if the rule
could not be read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Catch exceptions while reading the POSIX TZ rule

A failure to read the tzdata file should only mean panels fall back to the
manager's time, never break loading the config.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Each Sonoff panel gets a "Mute sounds" switch. While it is on, the
manager sends the panel's config without any buzzer sounds, which
silences every event; turning it off sends the configured sounds again.
No protocol or firmware change is needed.

The switch is registered with retain, so the broker keeps the last
command and the mute survives manager restarts. The retained command is
only cleared when the panel is removed.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* MQTTManager: send NSPanels the current time as Unix milliseconds

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MQTTManager: move the time and date thread into ManagerClock

publish_time_and_date() moves out of main.cpp unchanged, as
ManagerClock::_run_clock_thread(). ManagerClock::start() attaches the
MQTT connected callback and starts the thread, so main.cpp makes one call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MQTTManager: send the epoch with QoS 2

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* MQTTManager: mirror publish_if_connected() to STOMP while disconnected

STOMP is a full mirror of what is published, so the message goes to STOMP
before the connected check rather than only after a successful MQTT send.
The return value still means "sent over MQTT".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
… in case MQTT was not connected. Simply send all messages to all listeners on STOMP
…ufs (#410)

* Add media player sources to the NSPanel protobufs

Home Assistant media players have a list of sources (inputs) and can
select one. A media player with a source volume strategy also has a
source player feeding it, which has its own sources, for example the
Sonos Favorites of the speaker an audio matrix zone is playing from.

NSPanelEntityState.MediaPlayer gets both lists as a SourceSelection
(sources, current source and whether one can be selected).
source_player_sources is unset when there is no source player.

MediaPlayerCommand selects a source from either list by name. The lists
change when the player switches source, so an index from an older state
could select the wrong one. An empty name means no change, as no source
is called "".

The firmware C files were also missing volume_step from #385 and pick
it up here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add media playback position to the NSPanel media player protobuf

NSPanelEntityState.MediaPlayer gets the playback position and duration
of the current media, so panels can show a progress bar.

Home Assistant only reports the position when playback starts, stops or
seeks, together with the time it was taken. Sending that time as well
lets a panel advance the position itself while playing, without the
manager publishing a state every second. The state is retained, so a
panel can receive it long after it was sent; the time is absolute (Unix
seconds) rather than relative to when the state arrived for that reason.

has_media_position is unset when there is no position, and a duration
of 0 means it is unknown, as for a live stream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Add media player sources to the NSPanel protobufs

Home Assistant media players have a list of sources (inputs) and can
select one. A media player with a source volume strategy also has a
source player feeding it, which has its own sources, for example the
Sonos Favorites of the speaker an audio matrix zone is playing from.

NSPanelEntityState.MediaPlayer gets both lists as a SourceSelection
(sources, current source and whether one can be selected).
source_player_sources is unset when there is no source player.

MediaPlayerCommand selects a source from either list by name. The lists
change when the player switches source, so an index from an older state
could select the wrong one. An empty name means no change, as no source
is called "".

The firmware C files were also missing volume_step from #385 and pick
it up here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add media playback position to the NSPanel media player protobuf

NSPanelEntityState.MediaPlayer gets the playback position and duration
of the current media, so panels can show a progress bar.

Home Assistant only reports the position when playback starts, stops or
seeks, together with the time it was taken. Sending that time as well
lets a panel advance the position itself while playing, without the
manager publishing a state every second. The state is retained, so a
panel can receive it long after it was sent; the time is absolute (Unix
seconds) rather than relative to when the state arrived for that reason.

has_media_position is unset when there is no position, and a duration
of 0 means it is unknown, as for a live stream.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Send media player sources to NSPanels and let panels select them

The manager now reads source_list and source from Home Assistant and
sends them to the NSPanel with the media player state. Select commands
from the panel call media_player.select_source.

Sources of the source player (the player feeding this one, as set up
with the source volume strategy) are sent too. Only the source_entity
strategy used to follow the source player in Home Assistant, the
player_attributes strategy only read attributes of the player itself
and so could not see the source player's sources. Following the source
player now lives in HomeAssistantSourcePlayer, which both strategies
use. They only differ in where the source volume comes from. When the
player switches source, the sources of the previous source player are
dropped straight away rather than offered until the new ones arrive.

The NSPanel can only receive 4 KB in one MQTT message, so at most 32
sources of up to 64 bytes each are sent per list. Longer names are
shortened without splitting a UTF-8 character, and the shortened name
the panel sends back is matched to the full one before selecting it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Test media player source selection

Covers publishing a player's sources from Home Assistant, selecting a
source from the panel, ignoring sources that are unknown or can't be
selected, and limiting source lists to 32 names of at most 64 bytes
without splitting a UTF-8 character (selecting a shortened name sends
the full name to Home Assistant).

Also covers a source player feeding the media player: its sources are
published and selected on the source player, and switching to another
source player drops the previous one's sources.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Test media players with states recorded from Home Assistant

The media player tests use hand written Home Assistant states, which are
always tidy. Real ones are not, so this adds seven states recorded from
a real Home Assistant (personal names and the album art token replaced)
and checks what the manager publishes for the NSPanels from each:

- An audio matrix zone fed by a Sonos player: the zone's inputs, the
  Sonos favorites as source player sources, and the source volume. The
  Sonos player is playing a station that is not a favorite, so it has
  no current source. A favorite with a curly apostrophe is selected by
  its exact name.
- An Apple TV with 41 apps: only the first 32 reach the panel, and as
  the list is alphabetical that leaves out YouTube and TV.
- A receiver that is off, a matrix controller in an unknown state, and
  Music Assistant players with and without a source list.

The recorded states include album art, so the tests set the manager
address and port that album art URLs are built from.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Add media players to the React add/edit entity wizard

The REST API could already save media players (PUT /rest/entities/media_players),
but the UI had no way to add one.

- A "Media player" type, controlled by Home Assistant only, as the API requires.
- The form asks whether the media player is standalone or part of a multi-room
  audio setup fed by a source player. Only a multi-room setup asks how Home
  Assistant reports the source player: as one attribute holding its entity ID,
  or as attributes holding both the entity ID and the source volume. Attribute
  names are suggested from the selected player's current attributes.
- Validation matches put_media_player_entity, and attributes the chosen setup
  doesn't use are cleared on save.
- A banner marks media players as beta, since not all the firmware changes
  they need are merged yet.
- dist/ rebuilt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add a React test harness and media player UI tests

The React components had no tests. Add Vitest with Testing Library and jsdom
(npm test), kept in its own vitest.config.ts so tests never run build-only
plugins, and a CI workflow modelled on the Django one.

Tests cover the media player form (each setup and the request it sends,
validation, editing, attribute suggestions, server errors) and the wizard
(Home Assistant is the only controller offered for media players).

jsdom is pinned to ^28.1, the newest release supporting the same Node versions
as Vite (^20.19.0 || ^22.12.0 || >=24); jsdom 30 needs Node >=22.22.
bun.lock is updated alongside package-lock.json; it was behind package.json,
so zod moves to the 4.4.3 package.json already asked for.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Test the media player REST endpoint, return 405 for other methods

Cover PUT /rest/entities/media_players with the shared entity tests plus the
media player rules: each source volume strategy and the attributes it needs,
trimming, volume step limits, keeping the volume step when it is left out,
Home Assistant as the only controller, and refusing to overwrite another
entity type.

The shared tests found that an unsupported method returned 403, where every
other entity endpoint returns 405.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Home Assistant media players report media_position, media_duration and
media_position_updated_at, which were ignored. They are now sent to the
panels with the rest of the media player state.

A position without a time it was taken at can not be advanced, so it is
only sent together with a valid media_position_updated_at. The position
and duration are cleared when Home Assistant stops reporting them, for
example when playback stops or the player turns off.

media_position_updated_at is parsed with date::parse from Howard
Hinnant's date library, added as a header-only Conan dependency.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Fix the undefined names ruff reports (F821)

#391 deleted the NSPanelRoomEntitiesPages component when the room editor
moved to React, but ten htmx views still built one to render their
response. Each raised NameError, so they returned 500 once they reached
that line. Nothing in the templates, static JS or React code calls them
any more, so remove them, their routes, and handle_entity_modal_result,
whose only job was to dispatch to one of them:

- partial_remove_entity_from_page_slot
- partial_save_edit_entities_page
- partial_move_entity
- partial_move_entities_pages
- partial_delete_entities_page
- create_entities_page_in_room
- create_or_update_{light,switch,button,scene}_entity
- handle_entity_modal_result

scenes_post looked lights up through a Light model that no longer
exists, so POST /rest/scenes failed for any request with light states.
LightState.light points at Entity, so look the light up there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Regenerate the firmware protobuf code

Two .proto changes regenerated the MQTTManager C++ code but not the
protobuf-c code in protobuf_firmware/ that the firmware builds against:

- #394 added the Sound message and NSPanelConfig.sounds
- #405 added NSPanelConfig.tz_posix

The protobuf check added later in this series caught both.
Generated with protoc 27.0 and protoc-gen-c 1.4.1, as CI does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add a Django checks workflow: ruff, system checks and migrations

docker/ruff.toml never chose which rules to run, so it got whatever the
installed ruff enabled by default. ruff 0.8 reports 102 findings on
docker/web; 0.16 reports 394, mostly from rules the project never chose.
List the long-standing defaults (E4, E7, E9, F) explicitly, and exclude
the generated migrations and protobuf modules.

The "Django checks" workflow has two jobs:

- ruff, pinned, failing on undefined names and syntax errors (F821,
  E9). Those found the eleven broken call sites fixed in the previous
  commit. The remaining findings, mostly unused imports, need a cleanup
  pass before all of F can be enforced.
- Django's system checks and makemigrations --check, so a model change
  without its migration fails on the pull request rather than reaching
  installs whose database would no longer match the code. Both pass
  today. settings.py keeps its secret key and database under /data, so
  the job creates it first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Build the Docker image on pull requests

The image was only ever built by release.yml, so a broken Dockerfile,
Python dependency or MQTTManager cross-compile surfaced when a release
was published. Add a "Docker build" workflow that builds the image
without pushing it: amd64 and aarch64 on pull requests and pushes, and
also armv7, armhf and i386 weekly and on manual runs.

The release build compiles every Conan dependency from source, which
takes about an hour. Make the Conan package cache a BuildKit cache mount
so it can be kept between builds, and save it with actions/cache through
buildkit-cache-dance, keyed per architecture on conanfile.py and
install_build_tools.sh. Release and local builds behave as before on a
cold cache; local rebuilds now reuse the packages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Choose which Web UI to build with a WEB_UI_REF build argument

The ui-builder stage always cloned NSPanelManager-Web-UI's default
branch. Add a WEB_UI_REF build argument that takes a branch, tag or
full commit hash, so a build can use the Web UI's devel branch or a
fixed commit. It defaults to main, so releases and existing builds are
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Check the generated protobuf code is up to date in CI

The C++, Python and firmware C code generated from docker/protobuf is
committed, so a .proto change can be merged without regenerating it, or
regenerated by only one side. Add a "Protobuf" workflow that regenerates
it and fails if the result differs from what is committed.

It also checks that protoc (install_protobuf.sh), the Conan protobuf
package and the Python protobuf package are the same release. The
generated C++ does not compile against a different runtime, and the
generated Python does not import with one. All three are 5.27.0 today.

Regenerating used to need the dev container, which takes a full Conan
build to create. docker/protobuf/regenerate.sh runs compile_all.sh in a
plain Ubuntu 24.04 container with protoc 27.0 and protoc-gen-c 1.4.1,
and needs nothing but Docker. The workflow runs that same script, so a
failing check is fixed by running it and committing the result, and the
error message says so. compile_all.sh's output directories can now be
set from the environment, with the same defaults.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Type-check and build the React components in CI

The React components' build output (dist/ and templates/scripts.html) is
committed and served as-is, so a change to src/ that isn't rebuilt, or
doesn't type-check, ships unnoticed. Add a "React components" workflow
that installs from package-lock.json, runs the build (tsc -b, then vite
build), and fails if the output differs from what is committed. The
committed build reproduces exactly today.

ESLint runs but does not fail the job yet: the existing code has 30
errors, mostly no-explicit-any and react-refresh/only-export-components.

package-lock.json is used rather than bun.lock because only the former
matches package.json: bun.lock predates the runtime dependencies added
since May, and bun install --frozen-lockfile fails on it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The existing reorder test has no panels, so it never rendered the panel
status badge that made the old response 500. This one gives a room a
panel; it fails if the reorder view goes back to re-rendering RoomsList.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Fix the rain chance on the screensaver always showing 0% (#203)

The current precipitation probability was never read from the forecast, so
it stayed 0. Request it from Open-Meteo and store it; a null (not every
model has one) shows 0% instead of throwing away the whole update.

The forecast days formatted precipitation_sum (mm) with a % sign, so 6.8 mm
showed as "7%". Show precipitation_probability_max, which was already
fetched but unused.

Adds tests that feed a recorded Open-Meteo response through the parser.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add the missing #define to weather.hpp's include guard

The header is included through entity_manager.hpp as well as directly by the
weather tests, which redefined the class.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…ng display_order (#417)

* Allow saving a panel in detached mode with no entity chosen (#80)

With no entities yet, the detached-mode entity field is empty, or missing
entirely because the browser leaves out an empty <select>. save_panel_settings
then did Entity.objects.get(id="") (or raised KeyError) and returned a 500.

Save detached mode without an entity instead. MQTTManager already handles
that state; it's the same as when a button's entity is deleted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Fix room_put ignoring display_order

room_put checked for "displayOrder" but read data["display_order"], so the
key GET /rest/rooms returns was silently ignored (and sending displayOrder
gave a KeyError 500). Check the key it reads. Removes the expectedFailure
from the test that pinned it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
On every turn on, HomeAssistantLight sent color_temp_kelvin, falling back to
COLOR_TEMP_MIN when the kelvin was unknown. Home Assistant reports no kelvin
for a light that is off, so after a manager restart lights came on at the
warmest white. It did this even for lights without colour temperature
support, and re-sent the kelvin on every brightness change.

Now a kelvin is only sent when the light supports colour temperature and the
kelvin is known. The "color temperature" turn on behaviour only forces it
when the light was off, and not when a new colour was picked. Also fixes the
hue/saturation check, which was missing brackets and could send hs_color to a
light in colour temperature mode.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…auth header (#386) (#418)

* OpenHAB thermostat: show the current temperature (#386)

The current temperature item was never attached as an observer (reload_config
detached it but never attached it), so panels showed 0.0°. Its callback also
compared events with the target temperature item, read the initial REST fetch
from "value" instead of "state", and never marked the reading available.

The initial fetch also set the target temperature's debounce timer, which
could drop a target change from OpenHAB in the following second; it no
longer does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* OpenHAB thermostat: keep the decimal on the initial target temperature (#386)

The target temperature fetched over REST at startup was rounded to a whole
degree (22.5 showed as 23). Round to one decimal, as websocket updates are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* OpenHAB thermostat: read the preset and swing item names the web app stores

The React UI stores openhab_preset_mode_item, openhab_swing_mode_item and
openhab_swingh_mode_item, but only the names the older settings form used
(openhab_preset_item etc.) were read. Presets and swing modes were commanded on
"openhab/items//command" and never followed OpenHAB. Read the new names,
falling back to the old ones for thermostats saved by the older form.

Also fixes the initial preset fetch comparing with the current HVAC mode, and
the horizontal swing fetch checking the vertical swing item's name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* OpenHAB thermostat: check the topic has an item before reading it

Each callback read topic_parts[2] before checking the size. OpenhabManager
already drops topics without an item, so this was not reachable from OpenHAB;
it just puts the check first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* OpenHAB scenes: keep the Authorization header alive for the request

activate() put fmt::format(...).c_str() of a temporary into the header list,
so the request was sent with a dangling pointer as its Authorization header.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Check which OpenHAB items were commanded, not how many commands were sent

Without optimistic mode each set_* call re-sends the earlier changes OpenHAB
hasn't confirmed yet, so three calls send six commands. The pinned test
assumed one per call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
cablesandcoffee and others added 30 commits November 9, 2026 20:36
…t have openhab_item_switch defined. Will now configure both item_switch and item_dimmer, which is used is controlled by can_dim in MQTTManager. issue #435
* Updated protobuf cover with position and tilt presets

* Updated manager, web interface and firmware to handle new cover page

* Fixed react build errors and bugs regarding inverted_positions

* Fixed NPM tests for cover

* Added invert_positions as a required field for rest cover api

* Fixed test fixtures and rexpected output

* Fixed remaining test issues
…CI (#407)

* Build every theme's TFT from one Editor compile, and check them in CI

Each region's four themes are the same HMI with different pictures, but
each was compiled separately in the Nextion Editor, driven by AutoHotkey
on Windows, with the pictures rendered by a desktop GIMP. Now only one
Editor compile per region is needed:

- make_pictures.sh renders every theme's pictures with GIMP in Docker,
  with no display, byte for byte the same as the committed pictures.
- make_tfts.sh builds all four themes from the one compiled TFT by
  swapping in each theme's pictures and recomputing the checksums
  (algorithm from https://github.com/UNUF/TFTTool). It also builds
  us_horizontal_mirrored from the EU TFTs, which differ only in the
  display orientation byte, so its AutoHotkey script and orientation
  reminder go.
- Image progress bars listed in progress_bars.txt get their pictures
  generated from the HMI instead of drawn by hand for every theme.
- hmi_tool.py shows an HMI's pages, components, event code and fonts,
  and edits numeric attributes without the Editor.
- check_build.py, run in CI with the tests, fails if the committed HMIs,
  pictures and TFTs don't agree.

The Nextion file formats come from nextion-tools, pinned in
requirements.txt. The README in tft_automation/ documents the workflow.

The committed TFTs are left as they are. The blinds page is only partly
compiled into them, so check_build.py and the tests fail on them until
they are rebuilt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* CONTRIBUTING: how to change the panel GUI

Committing an HMI builds nothing: one Editor compile per region is still
needed, and make_tfts.sh builds the other themes from it. List the steps
here, and point to the tft_automation README for the details.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Reworked initial setup to use new REST v2

* Fixed react build issues

* Rebuild with fixed issues

* Fixed a bug that caused the theme to not be saved

* Reverted some changed that were unintentional

* Rebuilt main.css
* Reworked initial setup to use new REST v2

* Fixed react build issues

* Rebuild with fixed issues

* Fixed a bug that caused the theme to not be saved

* Reverted some changed that were unintentional

* Rebuilt main.css

* Made it so that NSPanel internal state does not switch to "waiting" every reload_config call.
Made it so that firmware, data file and GUI md5 checksums are only calculated if the mtime of that file has changed
Fixed a bug where scenes regiered using the old HTMX setup were not loaded as they were defined as "nspm_scene" and the new ones are defined as "nspm"

* Updated test to reflect how NSPanel should behave and not as it did behave

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants